Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove specific model checks for general scitype check #731

Closed
wants to merge 2 commits into from

Conversation

pazzo83
Copy link
Collaborator

@pazzo83 pazzo83 commented Jan 25, 2022

First pass at simplifying type checks for machines - we simply check the expected scitype signature instead of specific checks for supervised, unsupervised, etc.

"using the syntax `machine(model, X, y)` or `machine(model, X, y, w)` " *
"while most other models with `machine(model, X)`. " *
"Here `X` are features, `y` a target, and `w` sample or class weights. " *
"In general, data in `machine(model, data...)` must satisfy " *
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line 102: Put "data" in back-quotes: "`data`".

"`scitype(data) <: MLJ.fit_data_scitype(model)` unless the " *
"right-hand side is `Unknown`. Here, the scitype of `args` " *
"in `machine(model, args...; kwargs)` does not match the scitype " *
"expected by model's `fit` method.\n" *
" provided: $S\n expected by fit: $F"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about we replace the last sentence (from the original string, which is using different notation) with:

"In the present case:\n"*
"scitype(data) = $S\n"*
"and\n"*
"fit_data_scitype(model) = $F"

"while most other models with `machine(model, X)`. " *
"Here `X` are features, `y` a target, and `w` sample or class weights. " *
"In general, data in `machine(model, data...)` must satisfy " *
"`scitype(data) <: MLJ.fit_data_scitype(model)` unless the " *
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can drop the qualifier MLJ. from MLJ.fit_data_scitype(model) as this is now exported.

@ablaom
Copy link
Member

ablaom commented Jan 25, 2022

@pazzo83 Regarding the failing log tests. Rather than something like the existing

(:warn, r"The scitype of `X`")

let's make this robust to changes to the doc-string by instead doing something like

(:warn, MLJBase.warn_generic_scitype_mismatch(S, F))

where S and F have been defined appropriately for the test.

Make sense?

@ablaom
Copy link
Member

ablaom commented Jan 26, 2022

Okay, after looking again at the code, JuliaAI/ScientificTypes.jl#182 is not an issue. I thought we were doing S = scitype(args), but we actually have

S = Tuple{elscitype.(args)...}
, which should avoid the problem.

So, I think your fail is what I first suggested: outdated logging tests. The error message thrown has changed, so we have to update the @test_logs, no?

@ablaom
Copy link
Member

ablaom commented Jan 26, 2022

Hang on. I think S = Tuple{elscitype.(args)...} should instead be S = Tuple{scitype.(args)...}, another bug.

@pazzo83
Copy link
Collaborator Author

pazzo83 commented Jan 27, 2022

Yeah I had originally changed it to scitype but I keep getting warnings that specify this difference:

scitype(data) = Tuple{CallableReturning{Table{AbstractVector{Continuous}}}, CallableReturning{AbstractVector{Multiclass{2}}}}

vs

fit_data_scitype(model) = Union{Tuple{Table, AbstractVector{<:Finite}}, Tuple{Table, AbstractVector{<:Finite}, AbstractVector{Union{Continuous, Count}}}}

@ablaom
Copy link
Member

ablaom commented Jan 27, 2022

@pazzo83 I'll have a proper look into this tomorrow, thanks.

@ablaom
Copy link
Member

ablaom commented Jan 27, 2022

@pazzo83 Looking into this now and making progress. Thanks for your continued patience.

@ablaom
Copy link
Member

ablaom commented Jan 28, 2022

Closing in favour of #732

@ablaom ablaom closed this Jan 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants